import numpy as np
from sklearn.datasets import load_sample_image, load_sample_images
from skimage import io
import cv2
# Visualisation libraries
## Text
from colorama import Fore, Back, Style
from IPython.display import Image, display, Markdown, Latex
## matplotlib
import matplotlib.pyplot as plt
from matplotlib.font_manager import FontProperties
from mpl_toolkits.axes_grid1.inset_locator import inset_axes
plt.style.use('seaborn-whitegrid')
import matplotlib as mpl
mpl.rcParams['figure.figsize'] = (17, 6)
%matplotlib inline
## seaborn
import seaborn as sns
import warnings
warnings.filterwarnings("ignore")
# HTML
from IPython.core.display import HTML
HTML("""<style>.output_png { display: table-cell; text-align: center; vertical-align: middle;}</style>""")
We use the sklearn image dataset to demonstrate basic operations using Open CV.
Images = load_sample_images()
Names = [x.split("\\")[-1].replace('.jpg','').title() for x in Images['filenames']]
Images = Images['images']
def ImShow(Images, Names, Title = 'Images', axis_tight = False):
fig, ax = plt.subplots(1, 2 , figsize = (17, 6))
ax = ax.ravel()
font = FontProperties()
font.set_weight('bold')
for i in range(2):
_ = ax[i].imshow(Images[i])
if axis_tight:
_ = ax[i].axis('tight')
_ = ax[i].axis('off')
_ = ax[i].set_title(Names[i], fontproperties=font, fontsize = 16)
_ = fig.subplots_adjust(wspace= 0.01)
if Title:
_ = fig.suptitle(Title, fontproperties=font, fontsize = 18)
return fig, ax
# Image Show
_,_ = ImShow(Images, Names, Title = 'Original Images')
There are low-pass filters (LPF) and high-pass filters (HPF) that can help denoising one-dimensional signals. Typically, a LPF can remove noise by blurring the image whereas a HPF can be helpful in finding edges in an image.
In OpenCV, filter2D is a filtering function that replies on convolving a kernel with an image.
cv2.filter2D(src, ddepth, kernel[, dst[, anchor[, delta[, borderType]]]]) → dst
Parameters:
ddepth – desired depth of the destination image; if it is negative, it will be the same as src.depth(); the following combinations of src.depth() and ddepth are supported:
kernel – convolution kernel (or rather a correlation kernel), a single-channel floating point matrix; if you want to apply different kernels to different channels, split the image into separate color planes using split() and process them individually.
To demonstrate this, we can consider the following example.
Example:
Img = Images[1].copy()
def ImgStd(Inp):
'''This function standardizes image values.'''
Out = np.round(Inp).astype(int)
Out[Out<0] = 0
Out[Out>255] = 255
return Out
# Generating a noisy image using mu = 0 and sigma = 0.8
mu = 0; sigma = 10
Img_noise = Img.copy() - np.random.normal(mu, sigma, size=Img.shape)
Img_noise = ImgStd(Img_noise)
fig, ax = plt.subplots(1, 2 , figsize = (17, 6))
ax = ax.ravel()
font = FontProperties()
font.set_weight('bold')
_ = fig.suptitle('Flower Image', fontproperties=font, fontsize = 18)
_ = ax[0].imshow(Img)
_ = ax[0].set_title('Original Image', fontproperties=font, fontsize = 12)
_ = ax[1].imshow(Img_noise)
_ = ax[1].set_title('Noisy Image', fontproperties=font, fontsize = 12)
for i in range(2):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i
# We can consider a kernal as follows
kernel = np.ones((8,8),np.float32)/64
Out = cv2.filter2D(Img_noise,-1, kernel)
Out = ImgStd(Out)
fig, ax = plt.subplots(1, 2 , figsize = (17, 6))
ax = ax.ravel()
_ = ax[0].imshow(Img_noise)
_ = ax[0].set_title('Noisy Image', fontproperties=font, fontsize = 12)
_ = ax[1].imshow(Out)
_ = ax[1].set_title('filter2D (Noisy Image)', fontproperties=font, fontsize = 12)
for i in range(2):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i
del Out, kernel
Image blurring is done by removing high-frequency contents (e.g: noise, edges). There are several ways to apply this.
Box blur (also known as a box linear filter) is a form of low-pass ("blurring") filter. In a box blur, each pixel is calculated using the average value of its neighboring pixels in the input image.
Moroever, Blur is a OpenCV function that blurs an image using the normalized box filter
cv2.blur(src, ksize[, dst[, anchor[, borderType]]]) → dst
Parameters:
The kernel function here:
$$\texttt{K} = \frac{1}{\texttt{ksize.width*ksize.height}} \begin{bmatrix} 1 & 1 & 1 & \cdots & 1 & 1 \\ 1 & 1 & 1 & \cdots & 1 & 1 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ 1 & 1 & 1 & \cdots & 1 & 1 \\ \end{bmatrix}$$Example:
Out = cv2.blur(Img_noise,(8,8))
fig, ax = plt.subplots(1, 2 , figsize = (17, 6))
ax = ax.ravel()
_ = ax[0].imshow(Img_noise)
_ = ax[0].set_title('Noisy Image', fontproperties=font, fontsize = 12)
_ = ax[1].imshow(Out)
_ = ax[1].set_title('blur (Noisy Image)', fontproperties=font, fontsize = 12)
for i in range(2):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i
del Out
To see this filter better, we can consider using an image with some text phrases.
Example:
Colors={'White':(255, 255, 255), 'Black':(0, 0, 0), 'Red':(255,0,0),'Green':(0,255,0), 'Blue':(0,0,255),
'Yellow':(255,255,0), 'Purple':(128,0,128), 'Indigo':(75,0,130),'OrangeRed':(255,69,0)}
Img2 = np.zeros((800, 800, 3), np.uint8)
Img2 = cv2.putText(Img2, text = 'Sample Text', org = (80, 300),
fontFace = cv2.FONT_HERSHEY_TRIPLEX, fontScale = 3, color = Colors['Yellow'], thickness = 2)
Img2 = cv2.putText(Img2, text = 'Sample Text', org = (80, 400),
fontFace = cv2.FONT_HERSHEY_TRIPLEX, fontScale = 3, color = Colors['Red'], thickness = 2)
Img2 = cv2.putText(Img2, text = 'Sample Text', org = (80, 500),
fontFace = cv2.FONT_HERSHEY_TRIPLEX, fontScale = 3, color = Colors['Green'], thickness = 2)
# Generating a noisy image using mu = 0 and sigma = 0.8
mu = 0; sigma = 50
Img2_noise = Img2.copy() - np.random.normal(mu, sigma, size=Img2.shape)
Img2_noise = ImgStd(Img2_noise)
fig, ax = plt.subplots(1, 2 , figsize = (17, 6))
ax = ax.ravel()
_ = fig.suptitle('Image with Texts', fontproperties=font, fontsize = 18)
_ = ax[0].imshow(Img2)
_ = ax[0].set_title('Original Image', fontproperties=font, fontsize = 12)
_ = ax[1].imshow(Img2_noise)
_ = ax[1].set_title('Noisy Image', fontproperties=font, fontsize = 12)
for i in range(2):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i
Out = cv2.blur(Img2_noise,(8,8))
fig, ax = plt.subplots(1, 2 , figsize = (17, 6))
ax = ax.ravel()
_ = ax[0].imshow(Img2_noise)
_ = ax[0].set_title('Noisy Image', fontproperties=font, fontsize = 12)
_ = ax[1].imshow(Out)
_ = ax[1].set_title('blur (Noisy Image)', fontproperties=font, fontsize = 12)
for i in range(2):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i
del Out
Box Filter is a OpenCV function that blurs an image using the box filter.
cv2.boxFilter(src, ddepth, ksize[, dst[, anchor[, normalize[, borderType]]]]) → dst
Parameters:
The function smoothes an image using the kernel:
$$\texttt{K} = \alpha \begin{bmatrix} 1 & 1 & 1 & \cdots & 1 & 1 \\ 1 & 1 & 1 & \cdots & 1 & 1 \\ \vdots & \vdots & \vdots & \vdots & \vdots & \vdots \\ 1 & 1 & 1 & \cdots & 1 & 1 \\ \end{bmatrix}$$where
$$\alpha = \begin{cases} \frac{1}{\texttt{ksize.width*ksize.height}}, & \mbox{when normalize=true},\\ 1, & \mbox{Otherwise}. \end{cases}$$Example:
fig, ax = plt.subplots(2, 2 , figsize = (17, 2*6))
ax = ax.ravel()
_ = ax[0].imshow(Img)
_ = ax[0].set_title('Original Image', fontproperties=font, fontsize = 12)
for i in range(1,4):
k = (2**(i-1))*10+1
_ = ax[i].imshow(cv2.boxFilter(src = Img, ddepth = -1, ksize = (k, k)))
_ = ax[i].set_title('boxFilter (Image)', fontproperties=font, fontsize = 12)
_ = ax[i].text(x = 0, y = int(0.02*Img.shape[1]), s = 'Kernel Size: %i by %i' % (k, k),
size=11, color = 'Navy', bbox=dict(boxstyle="square", ec='Navy',fc='LightSkyBlue'))
for i in range(len(ax)):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i, k
Gaussian function in one dimension can be defined as $$G(x)={\frac {1}{\sqrt {2\pi \sigma ^{2}}}}e^{-{\frac {x^{2}}{2\sigma ^{2}}}}$$ In two dimensions, it is the product of two such Gaussian functions, one in each dimension: $$G(x,y)={\frac {1}{2\pi \sigma ^{2}}}e^{-{\frac {x^{2}+y^{2}}{2\sigma ^{2}}}}$$ where x is the distance from the origin in the horizontal axis, y is the distance from the origin in the vertical axis, and σ is the standard deviation of the Gaussian distribution.
In OpenCV, Gaussian Blur is defined as follows.
cv2.GaussianBlur(src, ksize, sigmaX[, dst[, sigmaY[, borderType]]]) → dst
Parameters:
Example:
fig, ax = plt.subplots(2, 2 , figsize = (17, 2*6))
ax = ax.ravel()
_ = ax[0].imshow(Img)
_ = ax[0].set_title('Original Image', fontproperties=font, fontsize = 12)
for i in range(1,4):
k = (2**(i-1))*4+1
_ = ax[i].imshow(cv2.GaussianBlur(src = Img, ksize = (k, k), sigmaX = 0))
_ = ax[i].set_title('GaussianBlur (Image)', fontproperties=font, fontsize = 12)
_ = ax[i].text(x = 0, y = int(0.02*Img.shape[1]), s = 'Kernel Size: %i by %i' % (k, k),
size=11, color = 'Navy', bbox=dict(boxstyle="square", ec='Navy',fc='LightSkyBlue'))
for i in range(len(ax)):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i, k
The median filter is often used to remove noise from an image that preserves edges while removing noise. See A Fast Two-Dimensional Median Filtering Algorithm by Thomas S. Huang et al. for more details regarding this filter.
Median Blur is a OpenCV function that blurs an image using the median filter.
cv2.medianBlur(src, ksize[, dst]) → dst
Parameters:
Example:
fig, ax = plt.subplots(2, 2 , figsize = (17, 2*6))
ax = ax.ravel()
_ = ax[0].imshow(Img)
_ = ax[0].set_title('Original Image', fontproperties=font, fontsize = 12)
for i in range(1,4):
k = (2**(i-1))*10+1
_ = ax[i].imshow(cv2.medianBlur(Img, k, cv2.BORDER_DEFAULT))
_ = ax[i].set_title('medianBlur (Image)', fontproperties=font, fontsize = 12)
_ = ax[i].text(x = 0, y = int(0.02*Img.shape[1]), s = 'Kernel Size: %i by %i' % (k, k),
size=11, color = 'Navy', bbox=dict(boxstyle="square", ec='Navy',fc='LightSkyBlue'))
for i in range(len(ax)):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i, k
A bilateral filter is a non-linear, edge-preserving, and noise-reducing smoothing filter for images. The bilateral filter is defined as $$I^{\text{filtered}}(x)={\frac {1}{W_{p}}}\sum _{x_{i}\in \Omega }I(x_{i})f_{r}(\|I(x_{i})-I(x)\|)g_{s}(\|x_{i}-x\|).$$
where
See Bilateral Filtering for Gray and Color Images for more details about Bilateral Filtering.
In OpenCV, Bilateral Filtering is a function that applies the bilateral filter to an image.
cv2.bilateralFilter(src, d, sigmaColor, sigmaSpace[, dst[, borderType]]) → dst
Parameters:
Example:
fig, ax = plt.subplots(2, 2 , figsize = (17, 2*6))
ax = ax.ravel()
_ = ax[0].imshow(Img)
_ = ax[0].set_title('Original Image', fontproperties=font, fontsize = 12)
for i in range(1,4):
k = (2**(i-1))*10+1
_ = ax[i].imshow(cv2.bilateralFilter(src = Img, d = k, sigmaColor = 75, sigmaSpace = 75))
_ = ax[i].set_title('bilateralFilter (Image)', fontproperties=font, fontsize = 12)
_ = ax[i].text(x = 0, y = int(0.02*Img.shape[1]), s = 'Diameter = %i' % k,
size=11, color = 'Navy', bbox=dict(boxstyle="square", ec='Navy',fc='LightSkyBlue'))
for i in range(len(ax)):
_ = ax[i].set_aspect('auto')
_ = ax[i].axis('off')
del i, k
Moreover, for adaptive bilateral filter, see this article.